Papers with production systems
CFO: A Framework for Building Production NLP Systems (D19-3)
Copied to clipboard
Rishav Chakravarti, Cezar Pendus, Andrzej Sakrajda, Anthony Ferritto, Lin Pan, Michael Glass, Vittorio Castelli, J. William Murdock, Radu Florian, Salim Roukos, Avi Sil
| Challenge: | Using a new orchestration framework, we build, test, and deploy interactive NLP and IR systems to production environments. |
| Approach: | They introduce a new orchestration framework for building, experimenting with, and deploying interactive NLP and IR systems to production environments. |
| Outcome: | The proposed framework is well suited to a variety of use cases but is not suitable for academic benchmarking or industry specific use cases. |
Proximity-Based Multi-Turn Optimization: Practical Credit Assignment for LLM Agent Training (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing group-based policy optimization methods rely on statistical deviation within discrete batches, misallocating credit when task difficulty fluctuates. |
| Approach: | They propose a framework for multi-turn LLM agents that integrates global context . they propose GRPO, which integrates success-rate-aware modulation and proximity-based soft aggregation . |
| Outcome: | The proposed framework yields performance gains over existing baselines with negligible computational cost. |
When Speed Meets Intelligence: Scalable Conversational NER in an Ever-evolving World (2026.eacl-industry)
Copied to clipboard
| Challenge: | Large Language Models excel at understanding conversational semantics, but lack of data makes them impractical for production deployment. |
| Approach: | They propose a pipeline for generating multilingual conversational NER datasets with minimal human validation and a framework that leverages LLMs as semantic filters combined with catalog-based entity grounding to label live traffic data. |
| Outcome: | The proposed framework outperforms existing models on public and private conversations by 97.12% on CoNLL-2003 and 83.09% on OntoNotes 5.0. |
Defending Against Disinformation Attacks in Open-Domain Question Answering (2024.eacl-short)
Copied to clipboard
| Challenge: | Existing methods to defend against data poisoning attacks in open-domain question answering are lacking . |
| Approach: | They propose a method that uses query augmentation to find diverse passages that could answer the original question but are less likely to have been poisoned. |
| Outcome: | The proposed method provides gains of nearly 20% exact match across varying levels of data poisoning/knowledge conflicts. |
To Chat or Task: a Multi-turn Dialogue Generation Framework for Task-Oriented Dialogue Systems (2025.acl-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) are designed to handle complex task requests, but lack of specific datasets for training and evaluation of such systems . |
| Approach: | They propose a framework to generate a dataset for in-vehicle speech recognition systems . they train an in-car context sensor that correctly identifies the functional intent of the driver . |
| Outcome: | The proposed framework outperforms baseline models across experimental settings. |
Agent vs. Agent: Automated Data Generation and Red-Teaming for Custom Agentic Workflows (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing red-teaming frameworks like AgentHarm use static prompts and hardcoded toolsets . |
| Approach: | They propose a red-teaming framework that generates adversarial tasks and evaluation functions tailored to arbitrary toolsets and uses iterative prompt refinement with self-reflection to develop more effective attacks. |
| Outcome: | The proposed approach achieves 162% increase in attack success rate on o4-mini and 86% success on gemini 2.5 Pro. |
How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts. (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation systems are prone to gender biases in their learned representations. |
| Approach: | They propose to use contextual sentences to correct gender bias in Neural Machine Translation models. |
| Outcome: | The proposed method can be used to build better, bias-free translation systems. |
Imitation Attacks and Defenses for Black-box Machine Translation Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using simulated experiments, we demonstrate that MT systems can be stolen even when imitation models have different input data or architectures than their target models. |
| Approach: | They propose a defense that modifies translation outputs to misdirect optimization of imitation models. |
| Outcome: | The proposed defense degrades the adversary’s BLEU score and attack success rate at some cost in the defender’s performance and inference speed. |
D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation (2026.findings-acl)
Copied to clipboard
Junlin Li, Shuangyong Song, Guodong DU, Ngai Wong, Xuebo Liu, Yongxiang Li, Min Zhang, Jing Li, Xuelong Li
| Challenge: | Existing methods for fine-tuned large language models fail on fine-scale datasets . large data scale amplifies delta parameter magnitude, singular values, and entropy, causing compression errors. |
| Approach: | They propose a training- and data-free delta compression method that captures dominant delta structure and compensates residual low-rank approximation to recover fine-grained details from smaller residual error. |
| Outcome: | The proposed method outperforms existing methods on large-scale datasets on dense and MoE architectures. |
Analyzing and Modeling LLM Response Lengths with Extreme Value Theory: Anchoring Effects and Hybrid Distributions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches treat length as an incidental output property rather than a statistically regular phenomenon worthy of rigorous modeling. |
| Approach: | They propose a statistical framework for modeling and controlling large language model response lengths using extreme value theory and cross-validation on Qwen and DeepSeek architectures. |
| Outcome: | The proposed model improves tail fit and generalizability while maintaining generalizzability. |